Systems and methods for providing contact detection in an articulated arm

ABSTRACT

A sensing manipulator of an articulated arm is disclosed. The sensing manipulator includes a compliant section and a movement detection system provided along a first direction of the compliant section such that movement of the compliant section along both the first direction and at least one direction transverse to said first direction, are detectable by the movement detection system.

PRIORITY

The present application is a continuation of U.S. patent applicationSer. No. 17/123,862, filed Dec. 16, 2020; which is a continuation ofU.S. patent application Ser. No. 16/743,428, filed Jan. 15, 2020, nowissued as U.S. Pat. No. 10,913,159, on Feb. 9, 2021; which is acontinuation of U.S. patent application Ser. No. 16/545,627, filed Aug.20, 2019, now issued as U.S. Pat. No. 10,875,185, on Dec. 29, 2020;which is a continuation of U.S. patent application Ser. No. 16/212,113,filed Dec. 6, 2018, now issued as U.S. Pat. No. 10,618,177, on Apr. 14,2020; which is a continuation of U.S. patent application Ser. No.15/242,255, filed Aug. 19, 2016, now issued as U.S. Pat. No. 10,343,284on Jul. 9, 2019; which claims priority to U.S. Provisional PatentApplication Ser. No. 62/210,235, filed Aug. 26, 2015, the disclosures ofwhich are hereby incorporated by reference in their entireties.

BACKGROUND

The invention generally relates to robotic and other sortation systems,and relates in particular to articulated arm systems for use insortation systems.

Systems and methods using mechanical compliance to improve robotperformance during grasping and manipulation are known. Purpose-builtcompliant elements exist commercially that function as safety guards,such as, for example, position sensors sold by ABB Automation TechnologyProducts AB of Sweden. These devices may include magnetic breakaway orspring elements that deflect when contact between the robot and theenvironment is made. Additionally, these designs can include rudimentaryon/off sensing of a breakaway state, which is often used as a stopsignal to the robot controller.

More modern robotic systems in industry and academia have incorporatedflexible elements and deformation sensors in the joints of a robot arm(see for example, the Baxter Robot sold by Rethink Robotics, Inc. ofBoston, Mass. and the DLR Lightweight Robot III developed by theInstitute of Robotics and Mechanics at German Aerospace Center inGermany). Through the combined sensing of deformation at each joint, anapproximation of the force at the end-effector may be deduced. Such animplementation is undesirable in certain applications however (forexample, due to unnecessary added compliance that may degrade thepositional accuracy of the end-effector, added mechanical complexity andcost, and decreased payload capabilities of the robotic system), withthe added complication that any highly flexible end-effector on therobot arm causes the loads transmitted through to the joints to befairly small and difficult to reliably measure.

Force sensors are also known to be used in robotic manipulation systems.A typical force sensor consists of a rigid plate instrumented withseveral micro-scale deformation sensors such as strain gauges. Thisplate is commonly placed between the robot end-effector and the robotarm, and used to sense forces and torques acting on the end-effector.These sensors tend to be expensive and difficult to calibrate accuratelysince they measure deflections or strain on very small scales.Furthermore, a force sensor mounted between the end-effector and robotarm suffers from the issue mentioned above for joint-sensors, namelythat highly flexible elements on the end-effector will not createsignificant forces for detection at the force sensor.

There remains a need therefore for an improved sensing system forrobotic and other sortation systems.

SUMMARY

In accordance with an embodiment, the invention provides a sensingmanipulator of an articulated arm. The sensing manipulator includes acompliant section and a movement detection system provided along a firstdirection of the compliant section such that movement of the compliantsection along both the first direction and at least one directiontransverse to the first direction, are detectable by the movementdetection system.

In accordance with another embodiment, the sensing manipulator includesa compliant section providing movement of the compliant section in atleast two degrees of freedom, and a movement detection system providingoutput data regarding movement of the compliant section in the at leasttwo degrees of freedom.

In accordance with a further embodiment, the invention provides a methodof sensing the position an orientation of an object held by amanipulator at an end effector of a robotic system. The method includesthe steps of engaging the object in a working environment of the roboticsystem, perceiving an initial position of a movement detection system,lifting the object against gravity, and perceiving at least two of load,pitch, roll and yaw of the object with respect to the initial positionof the movement detection system.

BRIEF DESCRIPTION OF THE DRAWINGS

The following description may be further understood with reference tothe accompanying drawings in which:

FIG. 1 shows an illustrative diagrammatic function block view of arobotic system including a sensing manipulator system in accordance withan embodiment of the present invention;

FIGS. 2A and 2B show illustrative diagrammatic views of compliantinterfaces for use in various embodiments of the present invention;

FIG. 3 shows an illustrative diagrammatic isometric view of a sensingmanipulator in accordance with an embodiment of the present invention;

FIG. 4 shows an illustrative diagrammatic view of a sensing manipulatorin accordance with another embodiment of the present invention

FIGS. 5A and 5B show illustrative diagrammatic views of an end effectorin a system of an embodiment of the present invention engaging arelatively light object;

FIGS. 6A and 6B show illustrative diagrammatic views of an end effectorin a system of an embodiment of the present invention engaging arelatively heavy object;

FIGS. 7A and 7B show illustrative diagrammatic views of an end effectorin a system of an embodiment of the present invention engaging an objectthat presents an unbalanced load;

FIG. 8 shows an illustrative diagrammatic isometric view of a sensingmanipulator in accordance with a further embodiment of the invention inan extended state (not engaging an object);

FIG. 9 shows an illustrative diagrammatic isometric view of the sensingmanipulator of FIG. 8 in an engaged state (engaging an object);

FIG. 10 shows an illustrative diagrammatic view of a sensing manipulatorin accordance with a further embodiment of the invention that includesgrippers;

FIG. 11 shows an illustrative diagrammatic view of a sensing manipulatorin according accordance with a further embodiment of the invention thatincludes sensing jaws; and

FIG. 12 shows an illustrative diagrammatic enlarged view of a portion ofthe sensing manipulator of FIG. 11.

The drawings are shown for illustrative purposes only.

DETAILED DESCRIPTION

The invention provides in accordance with an embodiment, a novel sensingmanipulator that tracks the physical deformation of a robot end-effectoras it makes contact with an environment, including an object within theenvironment. Many robot end-effector designs rely on flexiblepassively-compliant elements that deform to accommodate the environment.This compliance is used to improve the quality and reliability ofcontact during grasping and manipulation, and to reduce the impact loadsapplied to both the robot and objects during contact.

The novel sensing manipulator discussed herein in accordance withcertain embodiments tracks these various modes of deformation, andprovides this information for use in higher-level automation software todetermine significant details about the state of end-effector contactwith the environment. This mode of sensing eliminates the need for anadditional complex mechanical element traditionally used to sense forcesor add compliance to a robot system, while minimally altering thestiffness and inertia of the pre-existing hardware. Placing the sensoras close as possible to the contact site, in accordance with anembodiment, ensures it is able to obtain signals relevant to themanipulation task unaltered by the dynamics of transmission through therobot structure.

In accordance with certain embodiments, sensing manipulators of thepresent invention may have several primary features with many ancillarybenefits, summarized here and discussed in more detail below.

The position deformation sensor design methodology provides A) a sensingstrategy that can sense the deformation of a compliant element alongmultiple axes simultaneously, B) a sensing system that can be applied toa variety of pre-existing compliant elements and eliminates the need fornew mechanical complexity along the serial chain of a robot arm, C) asensor solution that minimally affects the stiffness or inertia ofexisting compliant elements, and D) a sensor that is placed near theend-effector contact surface to obtain data that is both highlysensitive and is unaltered by the dynamics of force transmission throughthe robot.

The novel software and algorithms of certain embodiments of theinvention further provide A) software strategies that use the sensorinformation to detect the presence or absence of contact with the world,and B) software strategies that detect the amount of force and torqueimparted on the end-effector due to the external load of the object andgrasping configuration.

This general approach of deflection sensing and algorithms applied toprocess the resultant data, is illustrated via several examples asfollows. The design and methodology may be understood initially byconsidering a simplified illustration of the deflection sensor design asshown in FIG. 1. FIG. 1 shows a deformation sensor application diagramin accordance with an embodiment of the present invention, where thedeformation sensor is positioned adjacent the environment such that thesensing of the deflection sensor of FIG. 1 occurs at the point ofcontact with the environment.

In particular, the robotic system 10 includes a movement detectionsystem 12 such as a deflection sensor that is provided with a compliantinterface 14 such as a vacuum cup, for engaging an environment 16. Themovement detection system 12 and the compliant interface 14 are coupledto an end effector 18 attached to a robotic mass 20 of the roboticsystem. The compliant interface may be formed in a shape of a tubular orconical bellows using a flexible material as shown at 14 and 14 a inFIGS. 2A and 2B respectively. Note that the compliant interface may movein not only a direction as shown at A, but may also move in seconddirections shown at B (as shown) and D (into and out of the page) thatare transverse to the first direction, as well as directions as shown atC that are partially transverse to the first direction. Also note thecompliant interface is not necessarily a part of the deflection sensoritself, but may, in certain embodiments, be a natural part of themanipulation system.

The deformation sensor may be applied to systems where the deformationis not tightly constrained but rather provides multi-axis sensing,meaning that deformation may occur linearly, rotationally, or alongcomplex paths. The ability to allow for and sense this complexdeformation is a key differentiator from prior art systems. Severaltechnologies can be applied to provide sensors to the compliantinterface. It is important that this sensing not restrict or impede thecompliant motion, or add significant inertia or mass. Several sensorscould be applied to measure the deformation including but not limitedto; flex sensors (such as flex-sensitive resistors or capacitivesensors), magnetic field sensors (such as a compass or hall-effectsensors), or potentiometers.

FIG. 3 shows a sensing manipulator 30 in accordance with anotherembodiment of the invention wherein the sensing manipulator includes amovement detection system 32. The movement detection system 32 includesa static 3-axis magnetic field sensor 34 that is aligned against amagnet 36 attached to the central part of the compliant cup 38 by a ring50. A vacuum is provided at an open end 46 of the complaint cup 38. Asthe compliant cup 38 moves, so too does the ring 40. As the ring 40around the cup moves, so too does a bracket 42 as well as a magnet 46,which movement is detected with respect to the magnet sensor 44 attachedto the articulated arm 54 for sensing the axial flexure of the vacuumcup from which translations/roll/pitch/of the cup. When the magneticfield sensor is employed, the system may determine not only movements inthe elongated direction (x) of the deflection sensor with respect to thearticulated arm, but also movements in directions (y and z) that aretransverse to the elongated direction of the deflection sensor as wellas directions that are partially transverse to the elongated directionof the deflection sensor.

With reference to FIG. 4, in accordance with a further embodiment, thesystem may include an articulated arm 80 to which is attached an endeffector 82, again, which may be a tubular or conical shaped bellows.The end effector 82 also includes a sensor 84 that includes anattachment band 86 on the bellows, as well as a bracket 88 attached tomagnetic field sensor 90, and a magnet 92 is mounted on the articulatedarm 80. As the bellows moves in any of three directions (e.g., towardand away from the articulated arm as shown diagrammatically at A, indirections transverse to the direction A as shown at B, and directionspartially transverse to the direction A as shown at C. The magneticfield sensor 90 may communicate (e.g., wirelessly) with a controller 90,which may also communicate with a flow monitor 94 to determine whether ahigh flow grasp of an object is sufficient for continued grasp andtransport as discussed further below. In certain embodiment, forexample, the system may return the object if the air flow isinsufficient to carry the load, or may increase the air flow to safelymaintain the load.

FIGS. 5A and 5B show an object 160 being lifted from a surface 162 bythe end effector 82 that includes the load detection device of FIG. 5.Upon engaging the object 160, the system notes the position of thedetection device. Once the object 160 is lifted (FIG. 5B), the systemnotes the change in the sensor output. In this example, the loadprovided by the object 160 is relatively light. FIGS. 6A and 6B,however, show the end effector lifting a heavy object.

FIGS. 6A and 6B show an object 170 being lifted from a surface 172 bythe end effector 82 that includes the load detection device of FIG. 5.Upon engaging the object 170, the system notes the position of thedetection device. Once the object 170 is lifted (FIG. 6B), the systemnotes the change in the position of the detection device. As notedabove, in this example, the object 170 is heavy, presenting a higherload.

The system may also detect whether a load is not sufficiently balanced.FIGS. 7A and 7B show an object 180 being lifted from a surface 182 bythe end effector 82 that includes the load detection device of FIG. 4.Upon engaging the object 180, the system notes the position of thedetection device. Once the object 180 is lifted (FIG. 7B), the systemnotes the change in the position of the detection device. In thisexample, the object 180 presents a non-balanced load. The compliantelement may therefore, undergo substantial translational and angulardeformation.

Various further platform applications include the following. Thedeformation sensor concept is designed to integrate with existingpassive and active compliant components of a robot end-effector. In theabove embodiments, suction cups are used as examples of compliantmembers. Many different compliant elements however, could be used basedon the end-effector selected. In accordance with a further embodiment,the invention provides a movement detection system that includesforce-sensitive resistors. FIGS. 8 and 9, for example, show a sensingmanipulator 200 together with a vacuum cup 202 wherein the movementdetection system includes an array (e.g., three) of detectors 204 forsensing the axial flexure of the vacuum cup from whichtranslations/roll/pitch/of the cup can be deduced. In particular, theforce-sensitive resistors may include a conductive polymer that isprinted on a surface, wherein the conductive polymer changes itresistance in a predictable manner when a force is applied to thesurface. The sensing manipulator 200 may be attached to a robotic armvia a mounting element 208 (which couples to a robotic arm mount thatpasses between two of the detectors 204). A vacuum may be provided at anopen end 206 of the vacuum cup 202 for engaging an object 210 (as shownin FIG. 9).

Another such alternative compliant element example is the use of atwo-fingered robot gripper either at the wrist (as shown in FIG. 10) oron the finger tips (as shown in FIG. 11). Normally compliance is builtin at the fingertips or directly behind the wrist of the gripper. Adeflection sensor could easily be adapted to accommodate similaralternative designs. In particular, FIG. 10 shows a sensing manipulator220 in accordance with a further embodiment of the present inventionthat is attached to a robotic arm 222. The sensing manipulator 220includes a compliant section 224 and a sensing section 226 that includesa two finger gripper end effector 228. As shown at D and E, the sensingsection 226 may provide sensing of the position and orientation of theend effector 228 with respect to the robotic arm 222, e.g., by magneticor capacitive sensing.

FIG. 11 shows a sensing manipulator 230 that is attached to a roboticarm 232. The sensing manipulator 230 includes a gripper 234 thatincludes two jaws 236. On one or both jaws is provided a compliantelement 238, and on the compliant element 238 is provided a magnet 242.With further reference to FIG. 12 (which shows an enlarged view of aportion of one jaw 236) a corresponding magnetic sensor 240 is providedon the jaw. When the compliant element 238 is under a load (as shown bya force as shown at F), the sensor 242 will move respect to the sensor240, providing position and orientation sensing data.

The stiffness and sensitivity of the compliant material are alsoimportant considerations. Note from FIG. 1 that the location of sensingis along the preexisting compliant structure of the robot system. Thisallows a system using the deformation sensor to maintain its originalstiffness and compliance properties, unlike prior art solutions. Alsoimportant to note is the target location for the deformation sensor inthe system. The more distal the sensor is the closer it is to theinteraction point, where non-linear complicating effects from the robotare less significant.

The software may involve high-level automation software that uses thedata output from the deformation to make a series of important decisionsas follows.

Contact State

The most straightforward application of the sensor is thresholding thedeformation values from the sensor to detect when contact with the worldhas occurred. If any axis of deformation moves outside nominal levels,then robot motion can be stopped and appropriate gripping strategymotions may be executed (such as pushing more or less on the environmentas needed).

Pre-Grasp Adjustment

When approaching an object for grasping, a robot arm will often firstmake contact with the object by pushing into it (either intentionally orunintentionally). Compliance is often used in robotic systems byallowing the end-effector to passively re-adjust to the environment bybending against the contact point. By using the deformation sensor tosense this angle of deflection, and then actively controlling the robotto re-adjust and compensate for the deflection by re-positioning itself,grasps can be made more reliable and centered on the object.

Force Sensing

Given a model of how the compliant element deflects under load, thedeformation changes may be mapped to forces and torques on theend-effector. This may allow for a number of force-sensing strategies,such as force-guided insertions and grasps, and force-guided placementof objects on surfaces.

Post-Grasp Centerpoint Sensing and Adjustment

Similar to the above two points, after an object is grasped and lifted,gravitational effects will cause the robot end-effector to deflect underthe load. Depending on the location of the grasp point with respect tothe center-of-mass of the object, this may cause various deformations inthe compliant element of the end-effector. Also, a poorly chosen grasplocation on a heavy object can induce oscillations between the compliantcomponents and object. The deformation sensor would be capable ofsensing both these effects, and may be used to guide the robot to acceptor reject grasps and give important information about the direction ofthe misalignment.

Human and Robot Safety

Due to centripetal effects the end-effector is often the most dangerouspoint on a moving robot arm. During motions where no environmentalinteraction is expected the deformation sensor can be monitored forchanges and the robot stopped when unexpected events occur. Thedeformation has advantages over the more traditional joint-level orwrist-level safety guards on a robot, since it is designed into thelow-inertia low-mass endpoint of the robot, and has the potential torespond before any damage has been done to the robot, environment, orhuman obstacles.

The deformation sensing strategy presented here provides a frameworkthat allows sensitive high-resolution sensing of contact between a robotand its environment, while minimally altering the physical attributes ofthe robot's compliance. Given a model or properly tuned heuristics thesensor may be used to resolve important information for robot decisionmaking to improve manipulation task performance.

Those skilled in the art will appreciate that numerous modifications andvariations may be made to the above disclosed embodiments withoutdeparting from the spirit and scope of the present invention.

What is claimed is: 1.-25. (canceled)
 26. A programmable motion systemfor grasping and moving objects, said programmable motion systemcomprising: a vacuum end-effector coupled to a vacuum source andincluding a vacuum sensor providing information regarding a vacuumprovided at the end-effector; a movement detection system for providinginformation regarding movement of the end-effector with respect to theprogrammable motion system in at least two degrees of freedom, and acontroller for determining, responsive to both the information regardingthe vacuum provided at the end-effector and the information regardingmovement of the end-effector with respect to the programmable motionsystem whether the vacuum provided at the end-effector on an object issufficient for continued grasp and transport of the object.
 27. Theprogrammable motion system as claimed in claim 26, wherein the vacuumsensor detects air flow.
 28. The programmable motion system as claimedin claim 26, wherein the controller further determines whether to returnthe object to an input location.
 29. The programmable motion system asclaimed in claim 26, wherein the controller further determines whetherto increase a vacuum flow provided at the end-effector to maintain theobject for transport.
 30. The programmable motion system as claimed inclaim 26, wherein the programmable motion system includes an articulatedarm.
 31. The programmable motion system as claimed in claim 26, whereinthe at least two degrees of freedom include any of load, pitch, roll andyaw.
 32. The programmable motion system as claimed in claim 26, whereinthe detection system provides information regarding movement of theend-effector with respect to the programmable motion system in threedegrees of freedom.
 33. The programmable motion system as claimed inclaim 32, wherein the end-effector includes a tubular or conicalbellows.
 34. The programmable motion system as claimed in claim 33,wherein the movement detection system includes a magnetic field sensor.35. The programmable motion system as claimed in claim 34, wherein themovement detection system includes an attachment band on a tubular orconical bellows of the end-effector.
 36. A programmable motion systemfor grasping and moving objects, said programmable motion systemcomprising: a vacuum end-effector coupled to a vacuum source andincluding a vacuum sensor providing information regarding a vacuumprovided at the end-effector, a movement detection system for providinginformation regarding movement of the end-effector with respect to theprogrammable motion system in at least two degrees of freedom, and acontroller for determining, responsive to both the information regardingthe vacuum provided at the end-effector and the information regardingmovement of the end-effector with respect to the programmable motionsystem whether to increase a vacuum flow provided at the end-effector tomaintain a grasp of an object by the end-effector for transport.
 37. Theprogrammable motion system as claimed in claim 36, wherein the vacuumsensor detects air flow.
 38. The programmable motion system as claimedin claim 36, wherein the controller further determines whether to returnthe object to an input location.
 39. The programmable motion system asclaimed in claim 36, wherein the movement detection system includes amagnetic field sensor.
 40. The programmable motion system as claimed inclaim 39, wherein the movement detection system includes an attachmentband on a tubular or conical bellows of the end-effector.
 41. Theprogrammable motion system as claimed in claim 36, wherein theprogrammable motion system includes an articulated arm.
 42. Theprogrammable motion system as claimed in claim 36, wherein the at leasttwo degrees of freedom include any of load, pitch, roll and yaw.
 43. Theprogrammable motion system as claimed in claim 36, wherein the detectionsystem provides information regarding movement of the end-effector withrespect to the programmable motion system in three degrees of freedom.44. The programmable motion system as claimed in claim 43, wherein themovement detection system includes a magnetic field sensor.
 45. Theprogrammable motion system as claimed in claim 44, wherein the movementdetection system includes an attachment band on a tubular or conicalbellows of the end-effector.
 46. A method of grasping and movingobjects, said method comprising: providing vacuum to an end-effectorcoupled to a programmable motion system, said end-effector including acompliant portion; contacting an object with the compliant portion ofthe end-effector such that the compliant portion changes shape; graspingand lifting the object by the compliant portion of the end-effector;providing information from a movement detection system regardingmovement of the compliant portion with respect to the programmablemotion system in at least two degrees of freedom; and determining,responsive to the information regarding movement of the end-effectorwith respect to the programmable motion system whether the vacuumprovided at the end-effector on the object is sufficient for continuedgrasp and transport.
 47. The method as claimed in claim 46, wherein themethod further includes providing information regarding the vacuumprovided at the end-effector by a vacuum sensor.
 48. The method asclaimed in claim 47, wherein the vacuum sensor detects air flow.
 49. Themethod as claimed in claim 46, wherein the method further includesdetermining whether to return the object to an input location.
 50. Themethod as claimed in claim 46, wherein the method further includesdetermining whether to increase a vacuum flow provided at theend-effector to maintain the object for transport.
 51. The method asclaimed in claim 46, wherein the at least two degrees of freedom includeany of load, pitch, roll and yaw.
 52. The method as claimed in claim 46,wherein the method further includes determining information regardingmovement of the end-effector with respect to the programmable motionsystem in three degrees of freedom.
 53. The method as claimed in claim46, wherein the method further includes adjusting any of a position ororientation of the end effector and the programmable motion device toreduce any movement of the end effector with respect to the programmablemotion system.
 54. The method as claimed in claim 46, wherein the methodfurther includes thresholding deformation values upon contact with theobject.
 55. The method as claimed in claim 46, wherein the methodfurther includes adjusting a grasp position of the end effector on theobject.
 56. The method as claimed in claim 46, wherein the methodfurther includes detecting unexpected movement of end effector withrespect to the programmable motion system during movement of the object.